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Abstract 

A version of the classical secretary problem is studied, in which one is interested 
in selecting one of the b best out of a group of n differently ranked persons who 
are presented one by one in a random order. It is assumed that b > 1 is a 
preassigned number. It is known, already for a long time, that for the optimal 
policy one needs to compute b position thresholds, for instance via backwards 
induction. In this paper we study approximate policies, that use just a single 
or a double position threshold, albeit in conjunction with a level rank. We give 
exact and asymptotic (as n — > oo) results, which show that the double-level 
policy is an extremely accurate approximation. 

Keywords: Secretary Problem; Dynamic Programming; Approximate Policies 

1 Introduction 

The classical secretary problem is a well known optimal stopping problem from 
probability theory. It is usually described by different real life examples, notably 
the process of hiring a secretary. Imagine a company manager in need of a secretary. 
Our manager wants to hire only the best secretary from a given set of n candidates, 
where n is known. No candidate is equally as qualified as another. The manager 
decides to interview the candidates one by one in a random fashion. Every time he 
has interviewed a candidate he has to decide immediately whether to hire her or 
to reject her and interview the next one. During the interview process he can only 
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judge the qualities of those candidates he has already interviewed. This means that 
for every candidate he has observed, there might be an even better qualified one 
within the set of candidates yet to be observed. Of course the idea is that by the 
time only a small number of candidates remain unobserved, a recently interviewed 
candidate that is relatively best will probably also be the overall best candidate. 

There is abundant research literature on this classical secretary problem, for which 
we refer to Ferguson [2] for an historical note and an extensive bibliography. The 
exact optimal policy is known, and may be derived by various methods, see for 
instance Dynkin and Yushkevich [1], and Gilbert and Mosteller [3J. Also, many 
variations and generalizations of the original problem have been introduced and 
analysed. One of these generalizations is the focus of our paper, namely the problem 
to select one of the 6 best, where 1 < 6 < n is some preassigned number (notice that 
b = 1 is the classical secretary problem). Originally, this problem was introduced 
by Gusein-Zade [5], who derived the structure of the optimal policy: there is a 
sequence < s\ < S2 < • • • < < st+i = n — 1 of position thresholds such that 
when candidate i is presented, and judged to have relative rank k among the first i 
candidates then the optimal decision says 



i < s\ : continue whatever k is; 

stop if k < j 



Sj + 1 < i < sj + i (where j = 1, . . . , b) : < 



i = n : stop whatever k is. 



continue if k > j; 



Furthermore, [5] gave an algorithm to compute these thresholds, and derived asymp- 
totic expressions (as n —> oo) for the 6 = 2 case. Also Frank and Samuels [3] pro- 
posed an algorithm, and gave the limiting (as n — > oo) probabilities and limiting 
proportional thresholds Sj/n. 

The algorithms of [31 [5] are based on dynamic programming, which means that the 
optimal thresholds Sj, and the optimal winning probability are determined numer- 
ically. The next interest was to find analytic expressions. To our best knowledge, 
this has been resolved only for b = 2 by Gilbert and Mosteller [3], and for 6 = 3 
by Quine and Law [6]. Although the latter claim that their approach is applicable 
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to produce exact results for any b, it is clear that the expressions become rather 
untractable for larger b. This has inspired us to develop approximate results for 
larger b. 



We consider two approximate policies for the general b case: single-level policies, 
and double-level policies. A single- level policy is given by a single position threshold 
s in conjuction with a rank level r, such that when candidate i is presented, and 
judged to have relative rank k among the first i candidates, then the policy says 

i < s : continue whatever k is; 

{stop if k < r 
continue if k > r; 
i = n : stop whatever k is. 

A double-level policy is given by two position thresholds si < s 2 in conjuction with 
two rank levels r\ < r%, such that when candidate i is presented, and judged to have 
relative rank k among the first i candidates, then the policy says 

i < si : continue whatever k is; 

stop if k < r\ 



s\ + 1 < i < s 2 



S2 + l<i<n — 1 : 



continue if k > r±; 

{Stop if k < T2 
continue if k > r 2 ; 
i = n : stop whatever k is. 

We shall derive the exact winning probability for these two approximate policies, 
when the threshold and level parameters are given. These expressions can then used 
easily to compute the optimal single-level and the optimal double-level policies, i.e., 
we optimize the winning probabilities (under these level policies) with respect to 
their threshold and level parameters. The most important result is that the winning 
probabilities of the optimal double-level policies are extremely close to the winning 
probabilities of the optimal policies (with the b thresholds), specifically for larger 
b. In other words, we have found explicit formulas that approximate closely the 
winning probabilities for this generalized secretary problem. As an example, we 
present in Table Q] the relative errors in percentages for a few n, b combinations (a 



3 



Table 1: Relative errors (%) of the optimal double-level policies. 



n = 100 n = 250 n= 1000 



6 = 10 1.702 1.841 1.911 
6 = 25 0.036 0.066 0.084 



more extended table can be found in Section 0]). Our second contribution is the 
derivation of asymptotic results for our level policies as re — > oo. This means both 
for the (optimal) winning probabilities, and the (optimal) fractional thresholds Sj/n. 

The paper is organized as follows. Section [5] contains the derivation of the exact 
winning probability of the single- level policy, and the associated asymptotic results; 
Section [3] deals with the two- level policies. Finally, in Section 2] we demonstrate how 
accurate the approximate policies are performing. 

2 Single-level policies 

Before we consider the single-level polcies we first introduce some notation we use 
throughout this paper. The absolute rank of the z-th object is denoted by Xj, while 
the relative rank of the i-th object is denoted by Y{. Ranks run from 1 to n, and 
we say that rank i is higher than rank j when i < j. Moreover for natural numbers 
x and re, the falling factorial x(x — 1) ... (x — re + 1) is denoted by (x) n . Note that 
{x) n is the number of n-permutations of a set containing x elements which is also 
the number of different injective functions from {1,2,..., n} to {1,2,..., x}. It is 
easily seen that (x) n = re!(^) and thus we have 



2.1 Winning probability for single-level policies 

In this variant of the well-known secretary problem the objective is to pick one of 
the 6 best objects from n objects consecutively arriving one by one in the usual 
random fashion known from the classical problem. In this subsection we consider 
the performance of the class of so-called single-level policies which is determined 
by two integer parameters s (called the position threshold) and r (called the rank 
level). Following such a single-level policy objects are considered to be selected from 
position s + 1 and then the first one encountered with a relative rank higher or equal 




(1) 



(y)n CO ' 
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than r is picked. Moreover, we assume that if the first n — 1 items are not picked 
that then the last object is certainly picked independent of its relative rank Y n . Let 
7r = ir(s, r) be such a policy with < s < n — 1 and 1 < r < b. To analyze the 
performance of this class of policies an explicit expression for the probability -Pslp(tt) 
of success when applying the single-level policy tt = tt(s, r) will be obtained. Thus 
PsLp(ft) is the probability that an object is picked with absolute rank higher than or 
equal to b if policy tt is applied. Note: when we wish to express explicitly parameters 
(n, b, s, r) we denote it, otherwise we omit it. 

Theorem 1. For r = 1, 2, . . . , b we have that Pshp(^(s,r)) = ^ if < s < r — 1 or 

s = n — 1, and otherwise 

Before proving this expression we need two auxiliary results. 
Lemma 2. For s = 0, 1, . . . , n — 2 and i = s + 2, s + 3, . . . , n we have that 



(min{Y s+ i, Yg +2 , . . . ,^-i} > r) = < 



if s < r 

ifs>r. 



(i-i) 



Proof. Let A be the event min{y^ + i, Y s+ 2, ■ ■ ■ > Y-i} > r. If s < r then Y s+ i < r and 
thus ¥(A) = 0. For s > r, i = s+2, . . . , n— 1 we have that F(A) is the probability that 
the rankings 1, 2, . . . , r are contained in the first s positions of a random permutation 
of the numbers 1,2,...,* — 1. Thus ¥(A) is the number of distinct injective functions 
from {1, 2, . . . , r} to {1, 2, . . . , s} divided by the number of distinct injective functions 
from {1,2, ... ,r} to {1,2, ... ,i - 1}. Hence F(A) = □ 

Lemma 3. For i = s + l,s + 2, . . . ,n — 1 and r = 1, 2, . . . , b we have that 



HYi < r\X t = j) 



1 for j = 1,2,..., r 

(j-l\(n-j\ 

ELi (7% /orj=r + l,r + 2,...,6. 



Proof We have that Yi < Xi and thus P(l^ < = j) = 1 if j < r. Suppose that 
j G {r + 1, r + 2, . . . , b}. li Xi = j then = k for A; G {1, 2, . . . , r} if A; - 1 objects 
from the j — 1 objects with absolute ranking smaller than j are among the first i — 1 
objects. Thus P(Yi = k\Xi = j) = F(H = k — 1) where H is a hypergeometrically 
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distributed random variable with population size n — 1, sample size i — 1 and j — 1 
successes in the population. Hence 



P(*i < rpQ = j) = = fc - 1) = £ 



(fe-l) (i-k) 



(n-l 

k=l k=l 



iT-l) 

□ 



Proof. (Of Theorem [H) The case s = n — 1 is trivial because then -PslpW^ - 1 5 r )) = 
P(X„ <&) = £. 

The cases s = 0, . . . , r — 1 are also trivial because then for sure Y s+ i < r, and 
thus P SL p(vr(s,r)) = P(X S+1 <&) = £. 

Now consider the 'general' case. For i = s + 1, s + 2, . . . ,n and j = 1, 2, . . . , 6 let 
Aj be the event that Xj = j and policy 7r(s, r) picks the object at position i: 

Aj = {min{y s+ i, Y s+2 , Y^} > r,Y t < r, X t = j}. 

Thus, 

n b 

i=s+l j=l 
b n-l b b 

jr' = l i=S+2 jr' = l jr' = l 

Cases i = s + 1 and i = n are treated seperately. Notice that for A; < i the relative 
ranks Y/% are independent of both X, and thus (for s+2<i<n— 1) 

P(4) = P(min{y s+1 , y s+2 , . . . , y^x} > r , y { < r , x t = j) 

= P(min{y s+1 , Y s+2 , . . . , Y^t} > r) P(^ < r, X, = j) 

= P(min{F s+1 , Y s+2 , . . . , Y^} > r) F(Y t <r\X i= j) PpQ = j), 

with P(Xj = j) = —, and the other two factors were determined in Lemma [2] and 
Lemma [3l For i = s + 1 : 

P(^+i) = P(X S+1 = j, y s+1 < r) = F(Y s+ i < r\X s+1 = j) F(X S+1 = j), 

and then apply Lemma [3] while noticing that (s) r /(i — l) r = 1. For i = n: 

F(AJ) = P(min{y s+1 , Y s+2 , . . . , Y n ^} >r,X n = j) 

= P(min{F s+ i, Y s+2 , . . . , > r)¥(X n = j), 

and apply Lemma [2j □ 

We defer the comparison of the performance of single-level policies with the 
optimal policy to Section [H 
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2.2 Asymptotic results for single-level policies 

For any n S N the probability of successfully applying a single-level policy n(s, r) 
is given by ([2]). Moreover, by enumeration, marginal analysis, and/or dynamic 
programming the values of s and r maximizing -PslpC 71 "^, r)) may be obtained (see 
Section 2J, but in general the computation time increases if n gets larger. However, 
in the limit n — > oo the expression given by ([2]) may be simplified. In this section we 
will find asymptotic results on the performance of an important family of single-level 
policies for n — > oo. To obtain these asymptotic results we restrict to the family 
of single- level policies for which there exists some no € N , tq € {1,2, ... ,6} and 

< a < 1 such that r = r$ for all n > no and lim„_> 00 - = a. In other words 
we assume that the rank level r is fixed for large n while the position threshold 
s ~ an. This is motivated by numerical evidence from dynamic programming 
that optimal single-level policies have these asymptotical properties. Under this 
assumption the optimal values of tq and a should depend (only) on b. The idea 
is that by obtaining an asymptotic simplification of ([2]) for all positive integers 
b the corresponding asymptotical optimal values for ro and a could be obtained 
analytically. 

Theorem 4. Fix r £ {1, . . . , b}, and let s = an + o(n) (as n — > oo ), then 

n limpW ( vr ( ,,r))=^/ 1 l (r+ £ £ (j [" J) x k ~\l - x)^ ] dx. (3) 

Ja \ j=r+lk=l V J ) 

Proof. Consider the hyper geometric probability (tZj) /(^Ti ) i n the expression 
of the winning probability ([2]) (see also Lemma [3J . Swapping the parameters of 
sample size and the number of successes, we know that this probability is equal to 
Gfc-i) (i-jfc) /(i-i) • This is interpreted as the probability of finding k — 1 successes 
in a sample of size j — 1 when this sample is drawn without replacement from a 
population of size n — 1 containing a total number of i — 1 possible successes. We 
notice in expression ([2]) that both population size n — 1 and the total number of 

1 — 1 successes tend to infinity, proportionally, whereas the sample size j — 1 remains 
fixed. Hence, we may approximate the hyper geometric probability by a binomial 
probability: 

(i-\\(n-i\ , . k-1 / -\j-k 

\k-l)\j-k) /J-l \ M — 1\ (n-iy 



("Zi 1 ) V fc - V V«- 1 
- 1 



n 



V / V / N / (4) 

x^ 1 (1 - x) J '- fe + o(l), 
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where i, n — > oo such that (i — l)/(n — 1) — > x € (0,1). Notice that equivalently, 
i/ra — > x. 



Next, consider the ratio (s) r /(i — l) r : 



II , . II " ' 



1 + fc 

(5) 

na + o(lj fay , 



r-1 



fc=0 n 

Combining the two asymptotics (|4|) and ([5]), it is easily seen that the first term in 
the expression of the winning probability ([2]), i.e., the £ part, can be considered as a 
Riemann sum converging to the integral ([3]) . Finally, the last term in the expression 
of the winning probability ([2]) is clearly less than b/n, and thus converges to zero as 
n — > oo. □ 

Denote the integrand in expression ([3]) of the asymptotic winning probability by 
f(x). Thus, the expression is 

- p SL < p( 7r ( a ' r )) = ° r / f{x)dx. 

The function a G (0, 1) i— y Pg^](7r(a,r)) is unimodal concave, thus making it easy 
to find numerically the optimal a by solving the first-order condition using the 
bisection procedure. Hence, we can compare empirically the asymptotic optimal 
winning probability with (finite) optimal winning probabilities. As an example, we 
set b = 5,r = 3. Then we find a* = 0.5046 and P s ( ~ ) (vr(a*, r)) = 0.765697. Figured] 
shows the winning probabilities p!^p(7r(s* , r)), where s* = s*(r) denotes the optimal 
position threshold given rank level r and the number of candidates n, obtained by 
optimizing the P^ p (tt(s, r)) with respect to s (see Section [3]). 



3 Double-level policies 

A natural extension of the single-level policies is the class of double-level policies for 
the secretary problem where the objective is to pick one of the b best objects from n 
objects consecutively arriving one by one in the usual random fashion. Let be given 
two rank levels 1 < r\ < ri < b, and two position thresholds r\ < s± < S2 < n — 1 
(we discard the trivial cases of S2 = n which gives again a single-level policy, and 
si < r\ which leads to stopping at position s\ + 1). The double-level policy says 
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Convergence of win.prob. 



0.774 



0.77 



0.7657 



2000 4000 6000 8000 10000 
n 



Figure 1: Optimal finite and asymptotic winning probabilities for single-level policies 
with b = 5, r = 3. 



to observe the first si presented objects without picking any; next, from objects at 
positions s± + 1 up to S2 the first one encountered with a relative rank higher or 
equal than n is picked; if no such object appears, the first object at positions S2 + 1 
up to n — 1 is selected which has a relative rank of at least r 2 ; finally, if all these n — 1 
items are not picked, the last object is certainly picked independent of its relative 
rank Y n . Slightly abusing, we denote again by ir = tt(s, r) such a double-level policy 
and by -Pdlp(tt) its winning probability. 

Theorem 5. The double-level policy given by rank levels 1 < r\ < r<i < b, and 
position thresholds r\ < s± < S2 < n — 1 has winning probability 

ftLP(ir(*,r))= £ 



, (i — l) ri 
i=si+l v ' 1 



+ 




(si) ri (s 2 - ri) r 2 1 ^ ^ (6) 



«=s 2 +l v ; 2 

(fljri (gg ~ n)r 2 -n _^ 

(n — l) r2 n 

Proof. Similar to the proof of Theorem [H once we have established that 

( s l) ri ( s 2 r l)r2— r 



P(min{y si+ i, . . . ,Y S2 } > n; min{f S2+ i, . . . , > r 2 ) . 

(i - l) r2 

This can be proved as follows. Let ^4 be the event of concern, then P(^4) is the 
probability that the rankings 1,2,... , r\ are contained in the first s\ positions of 



9 



a random permutation of the numbers 1, 2, . . . , % — 1, and the rankings r% + 1, r\ + 
2, . . . , r<i are contained in the first S2 positions. However, any permutation for which 
1,2, ... ,n are contained in the first s\ positions, leaves S2 — r\ positions for rankings 
r\ + l,r\ + 2, ... , r2 in order to become a 'feasible' permutation; the remaining 
i — 1 — r2 rankings can be positioned arbitrary. For P(A) we divide the number of 
feasible permutations by the total number of permutations: 



P(A) 



Ol)n (52 ~ n)r 2 - ri (i-1- r 2 )\ _ (Sl)ri (fg ~ n)r 2 -n 

(i-1)! " (i-l) r2 

□ 



Theorem 6. Fix rank levels 1 < r\ < r 2 < b, and let s± = ct\n + o(n) and 
S2 = ct2n + o(n) fas n — > oo), where < a\ < a 2 < 1. T/ien 



'/>hiJ(*>- i(i -^) 



+ n', l n.'r , ' L / — | r 2 + > > | ',' , ) .>•'•-' (J - r | r/.r 

(7) 

Proof. Similar to the proof of Theorem 01 □ 

Denote the two integrands in expression (JT]) of the asymptotic winning probabil- 
ity by fi(x) and f2(x), respectively. Thus, the expression is 



J OL\ J Ct2 



f 2 (x)dx. 



The functions a% E (0,02) >-» P^ p (ir(a, r)) (keeping «2 fixed), and «2 G (c*i,l) i-> 
^DLp( 7r ( a ' r )) (keeping ai fixed) are unimodal concave, thus making it easy to 
find numerically the optimal a = (01,02) by solving the first-order conditions 
using the bisection procedure. Hence, we can compare empirically the asymp- 
totic optimal winning probability with (finite) optimal winning probabilities. As 
an example, we set b = 10, n = 2,r2 = 6. Then we find a\ = 0.3630, a\ = 
0.6446 and P D ^(vr(a*, r)) = 0.957643. Fi gure [2] shows the winning probabilities 
pjy^p(n(s* ,r)), where s* = (sj(r), s^r)) denotes the optimal position thresholds 
given rank levels r = (r\,r2) and the number of candidates n, obtained by optimiz- 
ing the pjy^p(ir(s,r)) with respect to s = (si,S2) (see Section H]). 
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Convergence of win.prob. 



0.963 



0.96 



0.9576 



2000 4000 6000 8000 10000 
n 



Figure 2: Optimal finite and asymptotic winning probabilities for double-level policies 
with b = 5, r\ = 2, r<i = 6. 



4 Numerical Results 

We can find numerically the optimal single-level policy for a given number of can- 
didates n, and a given worst allowable rank b, in a two-step approach as: 

max max Pslp(^(s, r)). 

r=l,...,b s=r,...,n—l 

Thus, in the first step, we fix also a rank level r (between 1 and b). The function 
{r, ... ,n — 1} — > -PslpC 71 "!') r )) is unimodal concave (this follows after a marginal 
analysis), and thus we can solve numerically for the optimal position threshold 
s* = s*(r), and the associated winning probability Pslp(k(s*, »"))■ The second step 
is simply a complete enumeration to determine 

max{P SLP (7r(s*,r)) : r = 1, . . . b}. 

However, it can be shown that the function {1, ...,6} — > Pslp("7t(s*(-), •)) is uni- 
modal, which yields a shortcut in the second step. To check our numerical results, 
we have constructed an alternative method to find the optimal position threshold 
s*(r), given n,b,r, namely by dynamic programming (see the Appendix for the 
details) . 

Similarly, in the case of double- level policies, we have constructed a two-step ap- 
proach, where the first step finds the optimal position thresholds s* = s*(n,r2) and 
s 2 = s 2( r i' r 2) f° r an y given pair of rank levels (n,r2), and its associated winning 
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probability Pohpi^is* , r)) (vector notation for s and r). Then a straightforward 
search procedure determines 



max max i 3 DLp(vr(s*, r)). 

ri=l,....6— 1 r2=ri+l,...,6 



Finally, as mentioned in the introductory section, dynamic programming can be 
applied easily to obtain the optimal (multi-level) policy [3j[5]. 



Table [2] gives the relative errors of the winning probabilities of the optimal single 
and double-level policies for n = 100, 250, and n = 1000, and for b = 5, 10, ... , 25, 
relatively to the corresponding optimal multi-level policies. The double-level policy 
gives extremely small errors for larger 6, up to very large population sizes n. Also 
we notice that the errors (for a given b) increase slightly as n increases. Finally, we 



Table 2: Relative errors (%) of the optimal single- and double-level policies. 









single-level 






double-level 








n = 100 


n = 250 n 


= 1000 


n = 100 


n = 250 n 


= 1000 


b = 


5 


10.630 


10.854 


10.965 


3.286 


3.331 


3.354 


b = 


10 


5.262 


5.674 


5.876 


1.702 


1.841 


1.911 


b = 


15 


2.095 


2.467 


2.658 


0.568 


0.686 


0.746 


b = 


20 


0.739 


0.996 


1.131 


0.155 


0.221 


0.258 


b = 


25 


0.239 


0.381 


0.464 


0.036 


0.066 


0.084 



have computed the optimal asymptotic winning probabilities of the level policies: 

max P^(n(a*,r)), 

r=l,...,b 

where a* = a*{r) is the associated proportional rank level given r, obtained by the 
procedure elaborated in Section [2l Similarly, for the double-level policies 

max max P£j£(ir(a*, r)) 

n=l,...,o— 1 r2=ri+l,...,6 

yields the optimal asymptotic winning probabilities. Table [3] summarizes our com- 
putations for a range of 6-values. Also we included the asymptotic results of the 
optimal (full) policy, given in Frank and Samuels [3] (t\ = lim n ^ooS*/n for the 
optimal position threshold). Again we see how accurate the approximations of the 
level policies are. Notice that the a* thresholds are not monotone in b, this is due 
to the discrete character of the levels r\ and r%. 
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Table 3: Asymptotics of the optimal multi-level, single- and double-level policies. 



b 


h 


P(tt) 


r 


a 


PsLp(Tr) 


ri 


T2 




a 2 


^dlp(vt) 


5 


0.3255 


0.860347 


3 


0.5046 


0.765697 


1 


4 


0.2996 


0.6559 


0.831420 


10 


0.3129 


0.976530 


4 


0.4692 


0.918487 


2 


6 


0.3630 


0.6446 


0.957643 


15 


0.3068 


0.995902 


6 


0.5152 


0.968786 


3 


9 


0.3960 


0.6822 


0.988265 


20 


0.3031 


0.999271 


7 


0.4990 


0.987504 


4 


12 


0.4164 


0.7051 


0.996561 


25 


0.3006 


0.999869 


9 


0.5270 


0.994938 


5 


14 


0.4304 


0.6965 


0.998961 



5 Conclusion 

For the considered generalized secretary problem of selecting one of the b best out of 
a group of n we have obtained closed expressions for the probability of success for all 
possible single- and double-level policies. For any given finite values of n and b these 
expressions can be used to obtain the optimal single-level policy respectively optimal 
double- level policy in a straightforward manner. Moreover, asymptotically for n — > 
oo we have also obtained closed expressions for the winning probability for relevant 
families of single-level and double-level policies. Optimizing this expression for the 
family of single-level policies an asymptotic optimal rank level r and corresponding 
optimal position threshold fraction a* and asymptotic winning probability are easily 
obtained. Similarly we have done such asymptotic analysis and optimization for the 
relevant family of double-level policies. Both for the single-level and double-level 
policies we confirmed numerically for 6=5 that the winning probabilities for optimal 
finite and double level policies for finite values of n converge if n increases to the 
(respectively single- level and double- level) optimal asymptotic winning probabilities. 

Finally, we computed for varying b and n the optimal single-level and double-level 
policies and corresponding winning probabilities and compared the results to the 
overall optimal policy which is determined by b position thresholds. We found that 
the single-level policies and especially the double-level policies perform nearly as 
well as the overall optimal policy. In particular for a generalized secretary prob- 
lem with a larger value of b applying the optimal single-level or double-level policy 
could be considered, because implementation of the overall optimal policy using b 
different thresholds is unattractive compared to using only one or two thresholds for 
implementing the policy. Besides for large b the gain in performance of the overall 
optimal policy over the optimal double-level policy is very small. 
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Appendix: Dynamic Programming 

The dynamic programming method might be applied to find numerically the optimal 
single-, double- and multiple-level ('full') policies. Here, we summarize the algorithm 
for the single-level policy; it is straightforward how to generalize the algorithm to 
the double-level, and the multiple- level cases. 

Define the single-level policy with threshold s and level r, denoted ir(s,r), by its 
actions 

i < s : ai(k) 
s + 1 < i < n — 1 : a,i(k) 
i = n : a n (k) 

where 1 means to continue, and means to stop and select this candidate. We 
restrict to 1 < r < b. Denote by -Pslp(tt(s, r)) the probability of winning when 
7r(s,r) is applied. Given level r we determine the optimal threshold s*(r), defined 
by 

s*(r) = argmax Pslp(tt(s, r)). 
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We use dynamic programming to find it. Define for i = 1, . . . , n — 1 the value 

to be the maximal probability of winning when Yi < r is observed, and fi(2) to be 

the maximal probability of winning when Yi > r. The optimality equations are: 



¥{Xn-i < b I y n -i < r), P(Y n <b) = - 
v ' v n. 

a=0 ^ 

a=l 



/n-i(l) = max < 



/n-x(2) = -; 
for i = n — 2,n — 3, . . . , r 

= max{ F(X t < 6 | Yj < r) , ^} 

a=0 a=l 

/,(2) = Tfy/mCl) + ^^/, + i(2), 
z + 1 z + 1 

and for i = r — 1, r — 2,...,1 (since then surely 1^ < r and l^+i < r): 

= max{ P(X, <6) , ^A+iO)} 

a=0 a=l 

/i(2) = not defined. 

One can show that the result of this DP recursion is indeed a SLP by setting s*(r) = 
max{z : a* = 1}. Moreover, note that probabilities P(Xj < b 1 1^ < r) occuring in 
the optimality equations can easily be obtained, for example by applying Lemma [3] 
and Bayes' rule. 
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